PCX Product Brief 

PEX 8664, PCI Express Gen 2 Switch, 64 Lanes, 16 Ports 


Features 

■ PEX 8664 General Features 

0 64-lane, 16-port PCIe Gen2 switch 

- Integrated 5.0 GT/s SerDes 

0 35 x 35mm 2 , 1156-ball FCBGA package 
0 Typical Power: 7.9 Watts 

■ PEX 8664 Key Features 
o Standards Compliant 

- PCI Express Base Specification, r2.0 
(backwards compatible w/ PCIe rl.Oa/1.1) 

- PCI Power Management Spec, rl.2 

- Microsoft Vista Compliant 

- Supports Access Control Services 

- Dynamic link-width control 

- Dynamic SerDes speed control 
0 High Performance 

♦ performance PAK 

V Read Pacing (bandwidth throttling) 

V Multicast 

V Dynamic Buffer/FC Credit Pool 

- Non-blocking switch fabric 

- Full line rate on all ports 

- Packet Cut-Thru with 176ns max packet 
latency (xl6 to xl6) 

- 2KB Max Payload Size 
o Flexible Configuration 

- Ports configurable as xl, x2, x4, x8, xl6 

- Registers configurable with strapping 
pins, EEPROM, I 2 C, or host software 

- Lane and polarity reversal 

- Compatible with PCIe 1.0a PM 
0 Multi-Host & Fail-Over Support 

- Configurable Non-Transparent (NT) port 

- Failover with NT port 

- Up to Five upstream/Host ports with 1+1 
or N+l failover to other upstream ports 

o Quality of Service (QoS) 

- Eight traffic classes per port 

- Weighted round-robin source 
port arbitration 

o Reliability, Availability, Serviceability 

♦ visionPAK 

V Per Port Performance Monitoring 

■ Per port payload & header counters 

V SerDes Eye Capture 

V Error Injection and Loopback 

- 4 Hot Plug Ports with native HP Signals 

- All ports hot plug capable thru I 2 C 
(Hot Plug Controller on every port) 

- ECRC and Poison bit support 

- Data Path parity 

- Memory (RAM) Error Correction 

- INTA# and FATAL_ERR# signals 

- Advanced Error Reporting 

- Port Status bits and GPIO available 

- Per port error diagnostics 

- JTAG AC/DC boundary scan 


The ExpressLane™ PEX 8664 device offers Multi-Host PCI Express 
switching capability enabling users to connect multiple hosts to their 
respective endpoints via scalable, high bandwidth, non-blocking 
interconnection to a wide variety of applications including servers, 
storage systems, and communications platforms. The PEX 8664 is 
well suited for fan-out, aggregation, and peer-to-peer applications. 

Multi-Host Architecture 

The PEX 8664 employs an enhanced version of PLX’s field tested PEX 8648 
PCIe switch architecture, which allows users to configure the device in 
legacy single-host mode or multi-host mode with up to five host ports 
capable of 1+1 (one active & one backup) or N+l (N active & one backup) 
host failover. This powerful architectural enhancement enables users to build 
PCIe based systems to support high-availability, failover, redundant and 
clustered systems. 

High Performance & Low Packet Latency 

The PEX 8664 architecture supports packet cut-thru with a maximum 
latency of 176ns (xl6 to xl6). This, combined with large packet memory, 
flexible common buffer/FC credit pool and non-blocking internal switch 
architecture, provides full line rate on all ports for performance-hungry 
applications such as servers and switch fabrics. The low latency enables 
applications to achieve high throughput and performance. In addition to low 
latency, the device supports a packet payload size of up to 2048 bytes, 
enabling the user to achieve even higher throughput. 


Data Integrity 

The PEX 8664 provides end-to-end CRC (ECRC) protection and Poison bit 
support to enable designs that require end-to-end data integrity. PLX also 
supports data path parity and memory (RAM) error correction circuitry 
throughout the internal data paths as packets pass through the switch. 


Flexible Configuration 

The PEX 8664’s 16 ports can be 
configured to lane widths of xl, x2, x4, 
x8, or xl6. Flexible buffer allocation, 
along with the device's flexible packet 
flow control, maximizes throughput 
for applications where more traffic 
flows in the downstream, rather than 
upstream, direction. Any port can be 
designated as the upstream port, which 
can be changed dynamically. Figure 1 
shows some of the PEX 8664’s 
common port configurations in legacy 
Single-Host mode. 



Figure 1. Common Port Configurations 
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The PEX 8664 can also be configured in Multi-Host 
mode where users can choose up to five ports as 
host/upstream ports and assign a desired number of 
downstream ports to each host. In Multi-Host mode, a 
virtual switch is created for each host port and its 
associated downstream ports inside the device. The 
traffic between the ports of a virtual switch is completely 
isolated from the traffic in other virtual switches. Figure 
2 illustrates some configurations of the PEX 8664 in 
Multi-Host mode where each ellipse represents a virtual 
switch inside the device. 
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The PEX 8664 also 
provides several ways to 
configure its registers. The 
device can be configured 
through strapping pins, 

I 2 C interface, host 
software, or an optional 
serial EEPROM. This 
allows for easy debug 
during the development 
phase, performance 
monitoring during the operation phase, and driver or 
software upgrade. 
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Figure 2. Common Multi-Host Configurations 


Dual-Host & Failover Support 

In Single-Host mode, the 
PEX 8664 supports a Non- 
Transparent (NT) Port, 
which enables the 
implementation of dual¬ 
systems for redundancy and 
failover capability. The NT 
allows systems to isolate 
host memory domains by 
presenting the 
processor subsystem 
endpoint rather than 
another memory 
system. Base address registers are used to translate 
addresses; doorbell registers are used to send interrupts 
between the address domains; and scratchpad registers 
(accessible by both CPUs) allow inter-processor 
communication (Figure 3). 




k Non-Transparent 


Port 


End 


End 


End 

Point 


Point 


Point 


as an 


Figure 3. Non-Transparent Port 


Multi-Host & Failover Support 

In Multi-Host mode, PEX 8664 can be configured with 
up to five upstream host ports, each with its own 
dedicated downstream ports. The device can be 
configured for 1+1 redundancy or N+l redundancy. The 


PEX 8664 allows the hosts to communicate their status 
to each other via special door-bell registers. In failover 
mode, if a host fails, the host designated for failover will 
disable the upstream port attached to the failing host and 
program the downstream ports of that host to its own 
domain. Figure 4a shows a two host system in Multi- 
Host mode with two virtual switches inside the device 
and Figure 4b shows Host 1 disabled after failure and 
Host 2 having taken over all of Host l’s end-points. 



Hot Plug for High Availability 

Hot plug capability allows users to replace hardware 
modules and perform maintenance without powering 
down the system. The PEX 8664 hot plug capability 
feature makes it suitable for High Availability (HA) 
applications. Four downstream ports include a Standard 
Hot Plug Controller. If the PEX 8664 is used in an 
application where one or more of its downstream ports 
connect to PCI Express slots, each port’s Hot Plug 
Controller can be used to manage the hot-plug event of 
its associated slot. Every port on the PEX 8664 is 
equipped with a hot-plug control/status register to 
support hot-plug capability through external logic via the 
I 2 C interface. 

SerDes Power and Signal Management 

The PEX 8664 supports software control of the SerDes 
outputs to allow optimization of power and signal 
strength in a system. The PLX SerDes implementation 
supports four levels of power - off, low, typical, and 
high. The SerDes block also supports loop-back modes 
and advanced reporting of error conditions, which 
enables efficient management of the entire system. 

Interoperability 

The PEX 8664 is designed to be fully compliant with the 
PCI Express Base Specification r2.0, and is backwards 
compatible to PCI Express Base Specification rl.l and 
rl.Oa. Additionally, it supports auto-negotiation, lane 
reversal, and polarity reversal. Furthermore, the PEX 
8664 is tested for Microsoft Vista compliance. All PLX 
switches undergo thorough interoperability testing in 
PLX’s Interoperability Lab and compliance testing at 
the PCI-SIG plug-fest. 
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performance AK™ 

Exclusive to PLX, performance ^AK is a suite of unique 
and innovative performance features which allows 
PLX’s Gen 2 switches to be the highest performing Gen 
2 switches in the market today. The performance PAK 
features consists of the Read Pacing, Multicast, and 
Dynamic Buffer Pool. 

Read Pacing 

The Read Pacing feature allows users to throttle the 
amount of read requests being made by downstream 
devices. When a downstream device requests several 
long reads back-to-back, the Root Complex gets tied up 
in serving that downstream port. If that port has a narrow 
link and is therefore slow in receiving these read packets 
from the Root Complex, then other downstream ports 
may become starved - thus, impacting performance. The 
Read Pacing feature enhances performances by allowing 
for the adequate servicing of all downstream devices. 

Multicast 

The Multicast feature enables the copying of data 
(packets) from one ingress port to multiple (up to 15) 
egress ports in one transaction allowing for higher 
performance in dual-graphics, storage, security, and 
redundant applications, among others. Multicast relieves 
the CPU from having to conduct multiple redundant 
transactions, resulting in higher system performance. 

Dynamic Buffer Pool 

The PEX 8664 employs a dynamic buffer pool for Flow 
Control (FC) management. As opposed to a static buffer 
scheme which assigns fixed, static buffers to each port, 
PLX’s dynamic buffer allocation scheme utilizes a 
common pool of FC Credits which are shared by other 
ports. This shared buffer pool is fully programmable by 
the user, so FC credits can be allocated among the ports 
as needed. Not only does this prevent wasted buffers and 
inappropriate buffer assignments, any unallocated 
buffers remain in the common buffer pool and can then 
be used for faster FC credit updates. 

visionPAK™ 

Another PLX exclusive, vzszonPAK is a debug 
diagnostics suite of integrated hardware and software 
instruments that users can use to help bring their systems 
to market faster. vzszonPAK features consist of 
Performance Monitoring, SerDes Eye Capture, Error 
Injection, SerDes Loopback, and more. 


Performance Monitoring 

The PEX 8664’s real time performance monitoring 
allows users to literally “see” ingress and egress 
performance on each port as traffic passes through the 
switch using PLX’s Software Development Kit (SDK). 
The monitoring is completely passive and therefore has 
no affect on overall system performance. Internal 
counters provide extensive granularity down to traffic & 
packet type and even allows for the filtering of traffic 
(i.e. count only Memory Writes). 


SerDes Eye Capture 

Users can evaluate their system’s signal integrity at the 
physical layer using the PEX 8664’s SerDes Eye 
Capture feature. Using PLX’s SDK, users can view the 
receiver eye of any lane on the switch. Users can then 
modify SerDes settings and see the impact on the 
receiver eye. Figure 5 shows a screenshot of the SerDes 
Eye Capture feature in the SDK. 



Error Injection & SerDes Loopback 

Using the PEX 8664’s Error Injection feature, users can 
inject malformed packets and/or fatal errors into their 
system and evaluate a system’s ability to detect and 
recover from such errors. The PEX 8664 also supports 
Internal Tx, External Tx, Recovered Clock, and 
Recovered Data Loopback modes. 

Applications 

Suitable for host-centric as well as peer-to-peer traffic 
patterns, the PEX 8664 can be configured for a wide 
variety of form factors and applications. 

Host Centric Fan-out 

The PEX 8664, with its symmetric or asymmetric lane 
configuration capability, allows user-specific tuning to a 
variety of host-centric applications. Figure 6 shows a 
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server design where, in a quad or multi processor 
system, users can assign endpoints/slots to CPU cores to 
distribute the system load. The packets directed to 
different CPU cores will go to different (user assigned) 
PEX 8664 upstream ports, allowing better queuing and 
load balancing capability for higher performance. 



t. PC/e Genl or PCIe Gen2 slots 


Figure 6. Host Centric Dual Upstream 


Embedded or Communications Systems 

The PEX 8664’s 64 lanes can come in handy for 
embedded or communications applications requiring 
heavy processing and/or connectivity to multiple 
endpoints. Figure 7a shows an embedded system where 
the PEX 8664 is being used to fan-out to eight endpoints 
using x8 and xl6 links. Figure 7b shows a 
communications system where the PEX 8664 is using 
xl6 downstream links to fan out to three CPUs which 
have been configured as endpoints. These CPUs will run 
as endpoints, conducting different processing tasks while 
the host CPU (connected to the PEX 8664 via a xl6 
upstream link) manages them. 



Host Failover 

The PEX 8664 can also be utilized in applications where 
host failover is required. In the application shown in 
Figure 8, two hosts may be active simultaneously and 
controlling their own domains while exchange status 


information through doorbell registers or I 2 C interface. 
The devices can be programmed to trigger fail-over if 
the heartbeat information is not provided. In the event of 
a failure, the surviving device will reset the endpoints 
connected to the failing CPU and enumerate them in its 
own domain without impacting the operation of 
endpoints already in its domain. 



Figure 8. Host Fail-Over 


N+l Fail-Over in Storage Systems 

The PEX 8664’s Multi-Host feature can also be used to 
develop storage array clusters where each host manages 
a set of storage devices independent of others. Users can 
designate one of the hosts as the failover-host for all the 
other hosts while actively managing its own endpoints. 
The failover-host will communicate with other hosts for 
status/heartbeat information and execute a failover event 
if/when it gets triggered (see Figure 9). 
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Software Usage Model 

From a system model viewpoint, each PCI Express port 
is a virtual PCI to PCI bridge device and has its own set 
of PCI Express configuration registers. It is through the 
upstream port that the BIOS or host can configure the 
other ports using standard PCI enumeration. The virtual 
PCI to PCI bridges within the PEX 8664 are compliant 
to the PCI and PCI Express system models. The 
Configuration Space Registers (CSRs) in a virtual 
primary/secondaiy PCI to PCI bridge are accessible by 
type 0 configuration cycles through the virtual primary 
bus interface (matching bus number, device number, and 
function number). 

Interrupt Sources/Events 

The PEX 8664 switch supports the INTx interrupt 
message type (compatible with PCI 2.3 Interrupt signals) 
or Message Signaled Interrupts (MSI) when enabled. 
Interrupts/messages are generated by PEX 8664 for hot 
plug events, doorbell interrupts, baseline error reporting, 
and advanced error reporting. 



Figure 10. PEX8664-16U8D BB RDK 


Development Tools 

PLX offers hardware and software tools to enable rapid 
customer design activity. These tools consist of a 
hardware module (PEX 8664 RDK), hardware 
documentation (available at www.plxtech.com) , and a 
Software Development Kit (also available at 
www.plxtech.com ). 


PEX 8664 Rapid Development Kit (RDK) 

The PEX 8664 RDK (see Figure 10) is a hardware 
module containing the PEX 8664 which plugs right into 
your system. The PEX 8664 RDK can be used to test 
and validate customer software, or used as an evaluation 
vehicle for PEX 8664 features and benefits. The PEX 
8664 RDK provides everything that a user needs to get 
their hardware and software development started. 

Software Development Kit (SDK) 

PLX’s Software Development Kit is available for 
download at www.plxtech.com/sdk . The software 
development kit includes drivers, source code, and GUI 
interfaces to aid in configuring and debugging the PEX 
8664. For more information, please refer to the PEX 
8664 RDK Product Brief. 

Both performance PAK and vzszonPAK are supported by 
PLX’s RDK and SDK, the industry’s most advanced 
hardware- and software-development kits. 


Product Ordering Information 


Part Number 

Description 

PEX8664-AA50BC F 

64-Lane, 16-Port PCI Express Switch, 
Pb-Free (35x35mm 2 ) 

PEX8664-16U8D BB 
RDK 

PEX 8664 Rapid Development Kit with 
x!6 Upstream and Six x8 Downstream 


PLX Technology, Inc. All rights reserved. PLX, the PLX logo, ExpressLane, 
Read Pacing and Dual Cast are trademarks of PLX Technology, Inc. All other 
product names that appear in this material are for identification purposes only 
and are acknowledged to be trademarks or registered trademarks of their 
respective companies. Information supplied by PLX is believed to be accurate 
and reliable, but PLX assumes no responsibility for any errors that may appear 
in this material. PLX reserves the right, without notice, to make changes in 
product design or specification. 


Visit www.plxtech.com for more information. 
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